Predicting Short-Term Variations in End-to-End Cloud Data Transfer Throughput Using Neural Networks

Predicting data transfer throughput of cloud networks plays an important role in several resource optimization applications, such as auto-scaling, replica selection, and load balancing. However, constant short-term variations make the prediction end-to-end a very challenging task. The parameters that affect can be categorized into three different areas: end-system characteristics (e.g., disk I/O bandwidth, CPU utilization), network latency, background traffic, bandwidth shaping mechanisms), dataset average file size, size). Although there are promising results literature using neural networks, datasets collected from layer devices memory-to-memory transfers where not considered part problem. Also, few studies use multivariate time series variables differ study to study. In this project, we Amazon Web Services (AWS) by conducting intra- inter-region between storage systems compute resources monitoring services. This is unique sense metrics addition both source destination systems. Different instance type regionality provide various settings making applicable types models. Our models predict one-step ahead with ~ 3.7% 6.1% error rate, outperforming least-correlated univariate empowered learning.